Search AI Products and News

Recently, Professor Zhai Jidong's team from the Institute of High Performance Computing at Tsinghua University, in collaboration with Tsinghua-affiliated innovative enterprise, Qingcheng Extreme Intelligence, announced the open-sourcing of Chitu, a groundbreaking large model inference engine. This innovative technology marks another significant breakthrough for China in the AI field, particularly in inference engine development. A core highlight of the Chitu engine is its ability to natively run FP8 precision models on non-Nvidia Hopper architecture GPUs and various domestic chips.

Magic搭 Community collaborates with vLLM and FastChat to provide efficient LLM inference and deployment services. Developers can use vLLM as the inference engine in FastChat, offering high-throughput model inference. FastChat is an open platform for training, serving, and evaluating LLM-based ChatBots. vLLM is an LLM service developed by researchers from the University of California, Berkeley, Stanford University, and the University of California, San Diego.